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APPARATUS FOR, AND METHOD OF, ENCODING INFORMATION INTO, 
AND DECODING INFORMATION FROM, A SEQUENCE OF MOVING IMAGES 

The present invention relates to an apparatus for, and method of, encoding information 
into and decoding information from a sequence of moving images. The technique of 
coding information into a sequence of video or cinematographic images is also known as 
electronic watermarking. The invention is therefore particularly, but not exclusively, 
suitable for electronic watermarking all types of media having moving images stored 
thereon. 



It is known to insert codes into images for the purpose of identifying the owner of the 
images. The images may be in the form of video transmissions, or video clips or stills 
transmitted across a telecommunication link. There are a number of known schemes for 
inserting identification codes into so called 'sync' periods in transmitted video images. 
15 More recently it has been proposed to insert identification codes into the image itself in 
such a way that the code cannot be detected by a human eye. Such schemes may suffer 
from the disadvantage that low pass filtering and other image processes, such as data 
compression, may remove the code or degrade it to an extent that it cannot be recognised. 

20 In video broadcasting, data can be in either digital or analogue form. Increasingly data is 
transmitted in digital form. One reason for this is because larger amounts of data can be 
transmitted in digital form than in analogue form. Another is because digital signals are 
less prone to interference and can easily be recovered by using error correction 
techniques if received images are distorted. However, as side effects of facile access to 

25 digital data, complaints of copyright infringement and for tampering or modification of 
content are increasing. 

All current schemes suffer from the disadvantage that coded digital information cannot 
easily be transformed and maintain its integrity. Low pass filtering and other processes, 
30 such as data compression, may occur as a result of image compression algorithms or 
transmission of audio signals across a telecommunication link. Such transforms may 
remove the code or degrade it to an extent where it cannot be recognised. 



In the Applicant's published International Patent Application WO- A 1-9625005 (Todd), 
there is described a method of coding data into an image. The method of coding and 
decoding information into an image, comprises: dividing the image into MxN blocks, 
selectively encoding (decoding) information into selected blocks in such a way as not to 
be visible to a human eye, wherein in a decoding stage, the decoding is synchronised to 
the occurrence of the blocks for analysis of image information. In a preferred 
embodiment the size of insertions and their positions are fixed by processing the image in 
a block-by-block manner, typically with a block size of 8 by 8 pixels. 

In MPEG compression standards, temporal redundancy in image sequences is reduced by 
block-based motion compensation. Any change in an image from its previous frame 
caused by object movement is reserved to recover the image in the process of image 
decompression. Whilst previous encoding techniques have been successful, sometimes 
embedded codes have not survived MPEG compression. 

The present invention arose to overcome this problem. 

According to a first aspect of the present invention there is provided an apparatus for 
encoding information into a moving image sequence, comprising: means for identifying a 
region in an image in the sequence, means for determining whether the said region is a 
moving or static region and means for inserting a code into at least said moving image 
region. 

According to a second aspect of the present invention there is provided a method of 
encoding information into a sequence of images; the sequence of images comprising: a 
first image and a subsequent image, the method comprising the steps of: locating an edge 
in the first image, locating a corresponding edge in the subsequent image, comparing 
relative positions of the corresponding edges in said first and second images, thereby 
identifying either a relatively static edge or two relatively moving edges; inserting a first 
code into said moving edge and inserting a second code into said static edge. 
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The invention thus solves the problem of preserving both static and moving encoded 
information during subsequent image compression. 

Preferably the apparatus and method identify moving, and therefore static edges, in an 
5 image sequence and treat moving and static edges as separate channels each carrying its 
own code(s). Hence identification information is encoded into both moving edges (which 
are not compressed to any extent) and static edges independently. Both channels may 
carry identical codes if required. Alternatively different codes may be inserted into 
moving or static edges. The code insertion technique of the present invention may be 
10 used to embed codes in moving edge pixels. 

Means is advantageously provided to preselect said relative moving edges so that a code 
can be inserted into switchable sequences. If there is no movement in a sequence of 
images, no code is inserted and a different sequence may be selected. 

15 

According to another aspect of the present invention there is provided apparatus for 
decoding information from an encoded moving image sequence, comprising: means for 
identifying a region in an image in the sequence, means for determining whether the said 
region is a moving or static region and means for recovering code from at least said 
20 moving image region. 

Corresponding to this further aspect of the invention, there is also provided a method of 
decoding information from an encoded moving image sequence. 

25 Preferably coded information is inserted into an image in so called strongly featured 
regions of an image in such a way that the code is resistant to image compression and/or 
low pass filtering, but is not visible to the human eye. Examples of strongly featured 
regions of images are textured regions or lines, or edges between two regions of different 
luminance or contrast. In such regions, it is possible to insert a relatively large amount of 

30 information without significantly altering the image. 

Information encoded into an image may be used for a variety of purposes, for example: 



i) to insert copyright or identification information into video clips or still images; 

ii) to monitor when advertisements or films are played in broadcasts, for monitoring 
royalty payment purposes; 

iii) to identify a master copy of a data storage medium, such as a CD or DVD or 
video disc or similar medium, from which pirated copies may be produced. 

Codes are preferably inserted in edges within an image. Edge regions are known to have 
masking properties because of the way the human visual system works. In particular, the 
local orientation of edges are important, and there are specific structures in the primary 
visual cortex of the human brain, for detecting the presence of an edge and its local 
orientation. 

Coded information is preferably inserted into an image so that it does not alter the local 
orientation of certain features. The insertions are preferably made along the length of a 
local section of edge. The insertions are preferably made as a 2D function, by using for 
example an ellipse which is aligned to the local orientation of the edge. 

There is correspondingly also provided a method of encoding information into, and 
decoding information from, a moving image sequence, having one or more of the 
aforementioned preferred features. 

An embodiment of the invention will now be described, by way of example only, with 
reference to the Figures, in which:- 

Figure 1 shows diagrammatically an embodiment of an encoder according to the 
invention; 

Figure 2 shows an embodiment of a decoder according to the invention; 
Figure 3 shows a sketch with edges of images in bold; and 

Figure 4 shows the sketch of Figure 3, with only moving edges highlighted in bold. 
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Referring to the Figures, Figures 1 and 2 show block diagrams of an encoder 10 and 
decoder 20 respectively. The invention will now be described in the encoding sequence 
with reference to Figure 1. The signal is split into two portions: a moving image signal 
22 and a static image signal 23. This can be according to a standard MPEG protocol or a 
5 different proprietary image analysis (compression) protocol. 

A first code is inserted into the static image signal and a second code is inserted into the 
moving image signal. These signals are then transmitted in separate channels. 

The moving edge encoding sequence is now described with reference to the following 
10 Equations 1 to 3 and Figures 3 and 4. 

Moving edges and static edges may be defined as the following: 

Let I x (x,y), I 2 (x,y), and I 3 (x,y) denote three consecutive images in an image 
15 sequence. Let E 2 (x 7 y) be an edge image from image I 2 (x,y). The moving edge image 
ME{x,y) is defined as follows: 

ME(x, y) = D l2 (*, y) • E 2 (x, y) • D 23 (x, y) - (Eqn 1) 
where D l2 (x 9 y) = \l x (x 9 y)- I 2 (x,y% D 23 (x 7 y) = \l 2 (x 9 y)~ I 2 (x,y)\. - (Eqn 2) 

20 

The static edge image SE(x,y) may be obtained using the following equation: 

SE(x,y) = E2 (x,y) if ME (x f y) = 0 - (Eqn 3) 

Once inserted the coded information is treated the same as a moving image by any 
25 compression algorithm or protocol such as MPEG. The static code is included in early 
sequences of a series of encoded images so that when compressed the encoded 
information is in tact. 

Brief reference will now be made to Figure 2, which depicts a decoder 20. Decoder 20 
30 receives encoded images signal 24. The image signal 24 is split into a moving image 
signal 26. Edge diffraction then occurs. Edge detector 27 detects edges in moving 
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images. Edge detector 28 detects edges in static images. Detected edge signals 29 and 
30 are subtracted one from another at subtractor 31 and a static edge is decoded at static 
edge decoder 32. Moving edges are decoded directly from a signal presented by moving 
edge detector 27 to the moving edge decoder 33. Static edge decoded signals 34 and 
moving edge decoded signals 35 are added at 36 to provide an electronic watermark 
signal which indicates for example, the origin of a signal or the owner of copyright in an 
image sequence or piece of video footage. 

The invention may be used to encode information onto all forms of recording media. 
These may include videotape, video disc, compact disc (CD or DVD), or any other form 
of video storage medium. Similarly the invention may be incorporated into video 
broadcasting systems, video editing equipment, video monitoring equipment, televisions, 
computers or any other piece of electronic equipment used to produce or view video 
images, including a video cassette recorder and set-top box. 

The invention has been described, by way of example only, and it will be understood that 
variation may be made to the embodiments described without departing from the scope 
of the invention. 
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CLAIMS 

1. Apparatus for encoding information into a moving image sequence, comprising: 
5 means for identifying a region in an image in the sequence, means for determining 

whether the said region is a moving or static region and means for inserting a code into at 
least said moving image region. 

2. Apparatus for decoding information from an encoded moving image sequence, 
10 comprising: means for identifying a region in an image in the sequence, means for 

determining whether the said region is a moving or static region and means for 
recovering code from at least said moving image region. 

3. Apparatus according to claim 1 or 2 wherein means is provided to identify 
15 moving and static edges and signals representative of said moving and static image 

regions are sent via first and second channels so that code is inserted into, or recovered 
from, either or both of said channels. 

4. Apparatus according to claim 1, or claim 3 when dependant on claim 1, wherein a 
20 first code is inserted into signals in a first channel and a second code is inserted into 

signals in said second channel. 

5. Apparatus according to claim 4 wherein the first and second codes are identical. 
25 6. Apparatus according to claim 4 wherein the first and second codes are different. 

7. Apparatus according to any claim 1 wherein code is inserted into a moving image 
region of a moving image sequence in such a way that the code is resistant to image 
compression. 

30 

8. Apparatus according to claim 1 wherein code is inserted into a region of a moving 
image so that the code is resistant to low pass filtering. 
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9. Apparatus according to claims 1, 7 or 8 wherein the code is inserted into 
boundaries between regions of different luminance, chrominance or contrast. 

10. Apparatus substantially as herein described and with reference to the Figures. 

11. Apparatus according to any of claims 1 to 10 which is included in video 
recording, video broadcasting, video viewing equipment; or a television receiver or a 
set-top box. 

12. A method of encoding information into, or decoding information from, a moving 
image sequence using the apparatus of claims 1 to 10. 

13. A method of encoding information into a sequence of images; the sequence of 
images comprising: a first image and a subsequent image, the method comprising the 
steps of: locating an edge in the first image, locating a corresponding edge in the 
subsequent image, comparing relative positions of the corresponding edges in said first 
and second images, thereby identifying either a relatively static edge or two relatively 
moving edges; inserting a first code into said moving edge and inserting a second code 
into said static edge. 

14. A medium storing a video or cinematographic image or sequence of images 
thereon, characterised in that a code has been inserted into selective portions of said 
images, according to the method of claim 12, when dependant on claims 1 or claims 4 to 
9, or claim 13. 
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ABSTRACT 



An apparatus for, and method of, encoding information into and decoding information 
from, a sequence of moving images, such as video images. 

5 

It is known to insert codes into images for purposes of identifying the owner of the 
images. However, existing schemes have suffered from the disadvantage that certain 
data compression techniques have removed or degraded the code so that the code cannot 
be later recognised. 

10 

The present invention solves the problem by identifying moving and static portions of an 
image in an image sequence and providing two separate channels, one for the moving 
portion of an image and the other for a static portion of an image. One or more codes are 
then inserted into both moving and static image channels, so that moving image 
15 sequences including the code are, preserved even after image compression or low pass 
filtering. 

(Figure 1 accompanies the abstract) 




THIS PAGE BLANK (osptoj 



3 

OQ 
CD 
C/3 











a 


O 


o 




f— K 

cd 




O OQ 


O 


W 






OQ 




CD 



3 



w 

o 
o 

CD 





THIS PAGE BLANK (uspto) 



2/3 



TO 

c 

s 

CD 

to 

a 

CD 
O 

o 
:d 










O 




< 


cod 


ing 


5' 


tn 




a* 


OP 



PAGE BLANK (USPTO) 



H-7-OO 



THIS PAGE BLANK (uspto) 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

Q IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 
§3(faded TEXT OR DRAWING 

□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXfflBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER; 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



MSP; 



